Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stats inner/inter zone network traffic for mpp tasks #9747

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

Stats inner/inter zone network traffic for mpp tasks #9747

wants to merge 17 commits into from
+309 −77

Conversation

yibin87
Copy link
Contributor

@yibin87 yibin87 commented Dec 27, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #9748

Problem Summary:

What is changed and how it works?

Inner/inter zone network traffic is priced differently in cloud, thus trace each mpp query's such info will help users to locate the most network-expensive query.
This PR stats thress kinds of network traffic:

  1. Exchange sender
  2. Exchange receiver
  3. Remote table reader using coprocessor request

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Currently, set same and different zone labels for tiflash proxy config, and hack DAGStorageInterpreter to always use remote coprocessor read. Then check the MPPTaskStatistics.cpp output to see if local/inner/inter zone label is set correctly.
    When tidb side change is merged, will check slow log and statements summary.
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Dec 27, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from yibin87, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed do-not-merge/needs-linked-issue labels Dec 27, 2024
@yibin87
Copy link
Contributor Author

yibin87 commented Dec 27, 2024

/hold

@ti-chi-bot ti-chi-bot bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 27, 2024
@yibin87
Copy link
Contributor Author

yibin87 commented Jan 2, 2025

/cc @SeaRise @JinheLin @windtalker

@ti-chi-bot ti-chi-bot bot requested review from JinheLin and windtalker January 2, 2025 02:24
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jan 2, 2025

@yibin87: GitHub didn't allow me to request PR reviews from the following users: SeaRise.

Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @SeaRise @JinheLin @windtalker

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@yibin87 yibin87 requested a review from JinheLin January 2, 2025 09:53
@yibin87
Copy link
Contributor Author

yibin87 commented Jan 3, 2025

/unhold

@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 3, 2025
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 3, 2025
yibin87 added 10 commits January 3, 2025 11:59
@yibin87
Copy link
Contributor Author

yibin87 commented Jan 3, 2025

/cc @xzhangxian1008

@ti-chi-bot ti-chi-bot bot requested a review from xzhangxian1008 January 3, 2025 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@windtalker windtalker Awaiting requested review from windtalker

@JinheLin JinheLin Awaiting requested review from JinheLin

@xzhangxian1008 xzhangxian1008 Awaiting requested review from xzhangxian1008

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Stats inner/inter zone network traffic for mpp tasks
2 participants
@yibin87 @JinheLin